Overherd: designing information visualizations to make sense of student's online discussions

Overherd is an information visualization tool that makes patterns of interaction and activity in online discussion boards visible so that instructors can more easily analyze their students’ contributions. Our system uses the forums tool from the Sakai learning management system (LMS) to create visual representations of online behavior. We describe the design and development of Overherd, and provide design recommendations for "mashups" that extend the functionality of LMS yet rely on existing data within these learning environments. Introduction and Motivation With the nearly ubiquitous use of Learning Management Systems (LMS) in higher education, instructors are increasingly turning to online discussion boards to augment classroom discussion – some even require that students participate to earn part of their grade. In this form of "blended learning" (Garrison & Kanuka, 2004), online discussion boards provide a number of advantages for both students and teachers (see, e.g., Xie, 2006), including improved student performance (Krentler & Willis-Flurry, 2005). However, discussion boards present a number of problems as well. For instance, making sense of the posts in an online discussion board can be difficult, even when the conversation is threaded (Kay, 2006). The general use statistics most boards make easily available, such as counts of posts or words, are poor measures of the quality of learning taking place in those discussions (Mazzolini & Madison, 2005) and incorporating online discussions into existing teaching practices can increase the course’s workload for both students and instructors (Brush, et al., 2002). In an effort to help instructors make sense of the discussions happening on a board and to ease their workload, we designed an information visualization tool called Overherd. We designed Overherd using the metaphor of herd behavior to visualize what students are talking about by providing different views that show aggregation patters of students, topic, and keywords. The goal of Overherd is to make it easier for instructors to get an overview of what their students are talking about in online discussions by displaying visualizations that include post content, author information, and connections among posts (e.g., replies). These visualizations allow instructors to answer questions such as, “Are students using new terms introduced in class?” and “What course concepts are students most interested in discussing?” Overherd enables instructors to check for students’ understanding and to diagnose topics that may need additional discussion during face-to-face instruction. Designing and Studying Overherd Overherd is an extension of a Sakai-based LMS that includes a Forums tool for online discussions. In designing Overherd, we had very specific users and goals in mind – instructors of large lecture courses who used online discussions to extend classroom instruction. By limiting our design space, we were able to build a flexible platform that can still be extended to meet the goals of additional users in the future. In order to ground our design, we interviewed four instructors about their use of online discussion boards and later discussed paper prototypes of our designs with them. Overherd’s goals are to • facilitate faculty exploration of their classes’ overall understandings of a concept or concepts, and • assess student contributions and understanding for grading purposes. In order to address these goals, Overherd displays visualizations that include both content (e.g., the text of individual posts) and context (e.g., author information, timestamps). Table 1 provides an overview of the specific needs each visualization is designed to address. Figure 1 shows the design of Overherd. Overherd will be evaluated in two stages: first, a pilot user study where we display data from a completed course and ask instructors to evaluate the tool and second, a field study where Overherd will be deployed in several courses. These studies will explore whether Overherd helps instructors make sense of their students’ online discussions and reduces the instructors’ workloads. Table 1: Addressing user needs with visualizations User Need Visualization Overview of topics being discussed Treemap organized by term or person Determine which students use which terms Term/person node diagram See lists of posts that meet instructor’s search criteria Clustered list window Read posts that meet criteria Prose view window ICLS 2010 • Volume 2